SHARD: A Framework for Sequential, Hierarchical Anomaly Ranking and Detection
نویسندگان
چکیده
This work explores unsupervised anomaly detection within sequential, hierarchical data. We present a flexible framework for detecting, ranking and analyzing anomalies. The framework 1) allows users to incorporate complex, multidimensional, hierarchical data into the anomaly detection process; 2) uses an ensemble method that can incorporate multiple unsupervised anomaly detection algorithms and configurations; 3) identifies anomalies from combinations of categorical, numeric and temporal data at different conceptual resolutions of hierarchical data; 4) supports a set of anomaly ranking schemes; and 5) uses an interactive tree hierarchy visualization to highlight anomalous regions and relationships. Using both synthetic and real world data, we show that standard anomaly detection algorithms, when plugged into our framework, maintain a high anomaly detection accuracy and identify both micro-level, detailed anomalies and macrolevel global anomalies in the data.
منابع مشابه
A Hybrid Framework for Building an Efficient Incremental Intrusion Detection System
In this paper, a boosting-based incremental hybrid intrusion detection system is introduced. This system combines incremental misuse detection and incremental anomaly detection. We use boosting ensemble of weak classifiers to implement misuse intrusion detection system. It can identify new classes types of intrusions that do not exist in the training dataset for incremental misuse detection. As...
متن کاملImproving Shard Selection for Selective Search
The Selective Search approach processes large document collections efficiently by partitioning the collection into topically homogeneous groups (shards), and searching only a few shards that are estimated to contain relevant documents for the query. The ability to identify the relevant shards for the query, directly impacts Selective Search performance. We thus investigate three new approaches ...
متن کاملTraffic Scene Analysis using Hierarchical Sparse Topical Coding
Analyzing motion patterns in traffic videos can be exploited directly to generate high-level descriptions of the video contents. Such descriptions may further be employed in different traffic applications such as traffic phase detection and abnormal event detection. One of the most recent and successful unsupervised methods for complex traffic scene analysis is based on topic models. In this pa...
متن کاملCluster Based Anomaly Detection in Wireless LAN
Data mining methods have gained importance in addressing computer network security. Existing Rule based classification models for anomaly detection are ineffective in dealing with dynamic changes in intrusion patterns and characteristic. Unsupervised learning methods have been given a closer look for network anomaly detection. We investigate hierarchical clustering algorithm for anomaly detecti...
متن کاملHierarchical Group Compromise Ranking Methodology Based on Euclidean–Hausdorff Distance Measure Under Uncertainty: An Application to Facility Location Selection Problem
Proposing a hierarchical group compromise method can be regarded as a one of major multi-attributes decision-making tool that can be introduced to rank the possible alternatives among conflict criteria. Decision makers’ (DMs’) judgments are considered as imprecise or fuzzy in complex and hesitant situations. In the group decision making, an aggregation of DMs’ judgments and fuzzy group compromi...
متن کامل